Vehicle and trailer maneuver assist system

ABSTRACT

A trailer assist system for a vehicle includes a plurality of sensors disposed at a vehicle, a trailer sensor disposed at a rear portion of a trailer and having a field of sensing at least rearward of the trailer, and a control having a processor operable to process data captured by the sensors. A display screen displays an area rearward of the trailer. Responsive to a user input selecting a target area present in the displayed image, the control, via processing of data captured by the sensors, is operable to determine a path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward a target location represented by the selected displayed target area. The control, responsive to determination of the path of travel, controls at least steering of the vehicle to maneuver the vehicle and trailer along the determined path of travel.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the filing benefits of U.S. provisionalapplication Ser. No. 62/667,729, filed May 7, 2018, U.S. provisionalapplication Ser. No. 62/569,658, filed Oct. 9, 2017, and U.S.provisional application Ser. No. 62/549,989, filed Aug. 25, 2017, whichare hereby incorporated herein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates generally to a vehicle vision system for avehicle and, more particularly, to a vehicle vision system that utilizesone or more cameras at a vehicle.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporatedherein by reference in their entireties. It is also known to use camerasto assist in hitching a vehicle to a trailer and/or in determining atrailer angle of a trailer relative to a vehicle. Examples of such knownsystems are described in U.S. Pat. Nos. 9,085,261 and/or 6,690,268,which are hereby incorporated herein by reference in their entireties

SUMMARY OF THE INVENTION

The present invention provides a trailer maneuver assist system for avehicle that utilizes one or more cameras (preferably one or more CMOScameras) to capture image data representative of images exterior andrearward of the vehicle and trailer towed by the vehicle, and controlsthe vehicle to reverse the vehicle and trailer along a determined pathof travel and toward a target location. The system displays bird's-eyeview or surround view images of the vehicle and trailer and arearearward of the trailer and the user selects a target location. Thesystem determines an appropriate (obstacle free) path for the vehicleand trailer to follow to maneuver the trailer toward the selected targetlocation. The system provides control algorithms to assist maneuveringthe vehicle and trailer without the need for manual steering,acceleration or brake input by the driver.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle with a trailer maneuver assist systemthat is operable to control or guide the vehicle and to reverse thevehicle and trailer toward and into a selected location in accordancewith the present invention;

FIG. 2 is a block diagram of the functions of the trailer maneuverassist system of the present invention;

FIG. 3 is a block diagram of the algorithm flow of the trailer maneuverassist system of the present invention;

FIG. 4 is a view of the scene map and top view of the vehicle andtrailer as displayed by a display screen to allow the user to select thetargeted location;

FIG. 5 is a plan view of the vehicle and trailer showing pixel motionvectors for the trailer camera and the vehicle multi-camera system;

FIG. 6 is a plan view of a vehicle with another trailer maneuver assistsystem that is operable to control or guide the vehicle and to reversethe vehicle and trailer toward and into a selected location inaccordance with the present invention;

FIG. 6A is another plan view of a vehicle with another trailer maneuverassist system that is operable to control or guide the vehicle and toreverse the vehicle and trailer toward and into a selected location inaccordance with the present invention;

FIG. 7 is a block diagram of the functions of another trailer maneuverassist system of the present invention;

FIG. 8 is a block diagram of the algorithm flow of the trailer maneuverassist system of the present invention;

FIG. 9 is an image of a navigation screen showing a determined path froma current location of the vehicle and trailer to a user-selecteddestination;

FIG. 10 is a side elevation of the vehicle and trailer showing thesystem tracking a target in accordance with the present invention;

FIG. 11 is a view of a scene map and top view of the vehicle and traileras displayed by the display screen to allow the user to select currentand new targeted locations;

FIG. 12 is a view of another scene map and top view of the vehicle andtrailer as displayed by the display screen to allow the user to selectcurrent and new targeted locations;

FIG. 13 is a view of another scene map and top view of the vehicle andtrailer as displayed by the display screen to allow the user to select alocation and orientation; and

FIG. 14 is a view of another scene map and top view of the vehicle andtrailer as displayed by the display screen to allow the user to select anew location and orientation to replace a current location andorientation.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A vehicle and trailer maneuvering system or maneuver assist systemand/or driving assist system operates to capture images exterior of thevehicle and trailer being towed by the vehicle and may process thecaptured image data to determine a path of travel for the vehicle andtrailer and to detect objects at or near the vehicle and in thepredicted path of the vehicle, such as to assist a driver of the vehiclein maneuvering the vehicle and trailer in a rearward direction. Thevision system includes an image processor or image processing systemthat is operable to receive image data from one or more cameras and mayprovide an output to a display device for displaying imagesrepresentative of the captured image data. Optionally, the vision systemmay provide display, such as a rearview display or a top down or bird'seye or surround view display or the like.

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes a trailer maneuver assist system 12 thatis operable to maneuver the vehicle 10 and trailer 14 toward a desiredor selected location. The trailer maneuver assist system 12 includes atleast one exterior viewing vehicle-based imaging sensor or camera, suchas a rearward viewing imaging sensor or camera 16 (and the system mayoptionally include multiple exterior viewing imaging sensors or cameras,such as a sideward/rearward viewing camera 18, 20 at respective sides ofthe vehicle), and a rearward viewing trailer-based camera 22, whichcapture image data representative of the scene exterior of the vehicleand trailer, with the cameras each having a lens for focusing images ator onto an imaging array or imaging plane or imager of the cameras (FIG.1). Optionally, a forward viewing camera may be disposed at thewindshield of the vehicle and view through the windshield and forward ofthe vehicle, such as for a machine vision system (such as for trafficsign recognition, headlamp control, pedestrian detection, collisionavoidance, lane marker detection and/or the like). The trailer maneuverassist system 12 includes a control or electronic control unit (ECU) orprocessor that is operable to process image data captured by the cameraor cameras and may detect objects or the like and/or provide displayedimages at a display device for viewing by the driver of the vehicle (thecontrol and/or display device may be part of or incorporated in or at aninterior rearview mirror assembly of the vehicle, or the control and/orthe display device may be disposed elsewhere at or in the vehicle). Thedata transfer or signal communication from the camera to the ECU maycomprise any suitable data or communication link, such as a vehiclenetwork bus or the like of the equipped vehicle.

The system of the present invention is operable to automaticallymaneuver a trailer which is connected to a vehicle to a user's desiredor selected location. The system receives an input from a touch screenof the vehicle (such as at a center stack display) for the destinationlocation and converts the pixel (X, Y) coordinates from the touch inputto real world coordinates (XYZ). The system plans the path of thevehicle and trailer towards the desired location. The system may supportautonomous driving of the vehicle, and may automatically control thevehicle steering angle to direct the trailer toward the desired orselected location.

After initial selection of the target area displayed on the screen viathe user input, reversing of the vehicle and the trailer to the targetlocation (represented by the selected target area) is at leastsemi-autonomous in that at least steering, and preferably at leaststeering and the reversing speed of the vehicle, and more preferably atleast steering, reversing speed and braking of the vehicle, is underautomatic control by the control system. Further, image and/or sensordata captured by the vehicle and trailer sensors, during suchsemi-autonomous reversing maneuvering of the vehicle and trailer, isprocessed at a data processor using machine vision techniques, such asknown machine vision techniques. And should such data processing detectan object either in the determined or projected path of travel or likelyto enter the determined or projected path of travel (such as anothervehicle or pedestrian or bicyclist or the like present in or approachingthe determined or projected path of travel of the vehicle and trailer),the system reacts to mitigate any potential collision or hazard in suchsituations. For example, based on such a detected potential collision orhazard, an alert may be generated to a driver or occupant of the vehiclealerting that discontinuing the current reversing maneuver would berecommended. Optionally, for example, based on such a detected potentialcollision or hazard, the system may stop the maneuver itself and awaitfurther instructions/input from user. Optionally, for example, based onsuch a detected potential collision or hazard, the system may adjust therearward path of travel to mitigate or avoid impact with or hazard bythe detected object.

The system does not require user measurements and/or a target stickerattached to a trailer. The system may work with any type of conventionaltrailer such as vehicle haulers, box trailers, utility trailers, loadedor unloaded boat trailers, snow mobile trailers, and any other customtrailers. The system functions independent of lighting conditions (suchas, for example, day/night/sun/cloud/rain/snow) and road surfaces (suchas, for example, concrete/asphalt/gravel/grass/dirt). The system mayrequire an initial calibration where the vehicle is driving forward forless than five meters. However, no vehicle motion is preferred (nocalibration).

As shown in FIG. 1, the trailer maneuver assist (TMA) system includes acamera 22 at the rear end of a trailer (trailer camera), and amulti-camera system on the towing vehicle. The multi-camera systemincludes a rearward viewing camera 16 (such as a rear backup camera), aleft camera 18 (such as a side camera disposed at a left side rearviewmirror assembly of the vehicle), a right camera 20 (such as a sidecamera disposed at a left side rearview mirror assembly of the vehicle),and a center high mounted stop lamp (CHMSL) camera 24. The TMA system isoperable to perform without degradation, if at least the trailer cameraand rear view camera on the towing vehicle are present.

The algorithm for the TMA system is divided into three sections orblocks, as shown in FIG. 2. The system receives image data captured bythe cameras, and receives a touch input indicative of the targetlocation for the trailer, and performs an input analysis. Responsive tothe input analysis, the system plans the target path toward the targetlocation. The system includes a trailer control loop to control thevehicle to maneuver the vehicle and trailer toward the target location.The algorithm flow is shown in FIG. 3.

When the trailer maneuver assist (TMA) system is activated at a towingvehicle, the system checks if a trailer is connected to the towingvehicle, and the system checks if the connected trailer is already beencalibrated or prompts a list of previously calibrated trailer to theuser, such as at either an instrument cluster or center stack display ofthe towing vehicle, which allows the user to select a trailer from thelist. If the trailer is new (not previously calibrated), the system mayperform a one-time calibration for the trailer necessary toalign/calibrate the system for that trailer.

If the trailer is not calibrated, the system prompts a message, eitherat the vehicle instrument cluster or center stack display, to the userto calibrate the trailer with instructions displayed at either theinstrument cluster or center stack display to calibrate the trailer. Thecalibration instructions include “drive forward with/without turn”.During the calibration phase, the TMA system finds the relation betweenthe camera installed on the trailer and the towing vehicle, and measurespixel motion vectors (PMVs) of the trailer camera (T_PMV), and measuresthe PMVs for each camera in the multi-camera system of the towingvehicle (VChmsl_PMV, VLeft_PMV, VRight_PMV, VRear_PMV), such as shown inFIG. 5. The TMA system derives correlation matrices (CM[ ]) of thetrailer camera with each and every camera of the towing vehicle'smulti-camera system. Only a one time calibration sequence is requiredfor each new trailer. Once the calibration sequence is completed, theTMA system provides confirmation to the user on either the instrumentcluster or center stack display of the towing vehicle. The TMA systemstores the calibration values of the trailer into non-volatile memory ofthe hardware on which the TMA algorithm and software is running. ThePMVs include information about pixel speed (m/sec) in an orientation,and the maximum correlation matrix for the PMVs (MAX_CM_PMV) includesinformation pertaining to the maximum allowed pixel speed at the maximumallowed relative orientation of two cameras.

If the connected trailer has already been calibrated, or after thetrailer has been calibrated, the system will display the bird's eye viewof “towing vehicle+trailer” in a scene map on the display screen (suchas a display screen at a center stack) of the vehicle. The bird's eyeview or surround view is a top view projected image (comprising imagesderived from image data captured by the cameras of the vehicle andtrailer) that shows 360 degree surroundings of towing vehicle and thetrailer attached to it. The scene map (FIG. 4) is the region behind thetowing vehicle which covers ‘x’ meters along a longitudinal axis fromthe center of the rear view camera and ‘x/2’ meters at each side oflateral axis from the center of the rear view camera. The dimension ‘x’is preferably greater than 2× the length of the trailer, and thedimension ‘x’ depends on image resolution, field of view and height ofthe of the trailer camera. The scene map is generated using the trailercamera and the multi camera system on the towing vehicle. The TMA systemprompts a message to the user (such as at the center stack display) toselect a location in the scene map to which the trailer has to maneuver.The user may select a target location or point (X, Y) on the centerstack display either by touching the screen (if the screen is touchenabled) or by using cursor buttons to move a cursor to the targetlocation or point on the scene map.

When the user selects a location on the scene map to which the towingvehicle is to be maneuvered, the selected location in the screencoordinate system is then transformed into real world location (XYZ)using calibration parameters of the camera(s). The TMA system detectsavailable free space available in the scene map in which the towingvehicle and trailer can be maneuvered, where the free space is free ofobstacles. The TMA system identifies the path to maneuver towards thedesired location. As shown in FIG. 4, the vehicle and trailer in thescene map is shown in the displayed bird's eye view to the user. The TMAsystem initiates the maneuver towards the desired location once the useragrees with the path planned by TMA system. The TMA system stopsmaneuvering the vehicle and provides a message to the user whenever anobstacle enters into the planned path.

When the user selects a location on the scene map, the TMA systemautomatically calculates the distance between the rear end of thetrailer and the user's desired location (D_TrP), the path towards theuser's desired location, and the desired correlation matrix (DCM[ ])between the trailer camera and the vehicle multi-camera system for every1/500^(th) distance of D_TrP using pixel motion vectors (PMVs) of thetrailer camera and the vehicle multi cameras. When the maneuver isinitiated, the TMA system continuously calculates the currentcorrelation matrix (CCM) at a frequency of, for example, 60 samples persecond. If the CCM at a given point of time x is not matching withDCM[x] during the maneuver, then the trailer position will be adjustedby controlling the steering of the towing vehicle until the CCM matcheswith DCM values and the DCM will be updated as per the new position ofthe trailer. This step continues to happen until the vehicle reaches thedesired location. The TMA system considers the direction of maneuver toalign the pixel motion vector in the trailer camera with that of themulti-camera system of the towing vehicle. The system shall not crossthe maximum allowed pixel speed at the maximum allowed relativeorientation of two cameras (MAX_CM_PMV) at any given point of time.

Optionally, an advanced trailer assist (ATA) system may be similar tothe TMA system described above, but may use various sensors at thevehicle and trailer. For example, and such as shown in FIGS. 6 and 6A,the advanced trailer assist (ATA) system includes a plurality of sensors122 around or at the rear end of a trailer (trailer sensors), and amulti-sensor sensing system on the towing vehicle. The multi-sensorsensing system may include a plurality of rearward sensing sensors 116,a plurality of forward sensing sensors 117, a left sensor 118, a rightsensor 120, and a center high mounted stop lamp (CHMSL) sensor 124 (theside sensors and CHMSL sensor may comprise cameras and the forward andrearward sensors and the trailer sensors may comprise radar sensors,lidar sensors and/or ultrasonic sensors and/or the like). The multiplesensors at the trailer (and similarly at the rear of the tow vehicle andat the front/sides of the tow vehicle) may comprise a camera orimage-based sensor and a non-imaging sensor, such as a radar sensor,lidar sensor, an ultrasonic sensor and/or the like, such as shown inFIG. 6.

The algorithm for the ATA system is divided into three sections orblocks, as shown in FIG. 7. The system receives data sensed by thesensors, and receives a user input indicative of the target location forthe trailer, and performs an input analysis. Responsive to the inputanalysis, the system plans the target path toward the target location.The system includes a trailer control loop to control the vehicle tomaneuver the vehicle and trailer toward the target location. Thealgorithm flow is shown in FIG. 8.

When the ATA system is activated at a towing vehicle, the system checksif a trailer is connected to the towing vehicle, and the system checksif the connected trailer is already been calibrated or, if it is acalibrated trailer, the system prompts a list of previously calibratedtrailer to the user, such as at either an instrument cluster or centerstack display of the towing vehicle, which allows the user to select atrailer from the list. If the trailer is new (not previouslycalibrated), the system may perform a one-time calibration for thetrailer necessary to align/calibrate the system for that trailer.

If the trailer is not calibrated, the system prompts a message, eitherat the vehicle instrument cluster or center stack display, to the userto calibrate the trailer with instructions displayed at either theinstrument cluster or center stack display to calibrate the trailer. Thecalibration instructions include “drive forward with/without turn”.During the calibration phase, the TMA system finds the relation betweenthe sensors installed on the trailer and the towing vehicle, andmeasures motion vectors (MVs) of the trailer sensors (T_MV), andmeasures the MV for each sensor in the multi-sensor system of the towingvehicle (VChmsl_MV, VLeft_MV, VRight_MV, VRear_MV). The ATA systemderives correlation matrices (CM[ ]) of the trailer sensors with eachand every sensor of the towing vehicle's multi-sensor system. Only a onetime calibration sequence is required for each new trailer. And minimaleffort is needed to calibrate the trailer with the towing vehicle.

Once the calibration sequence is completed, the ATA system providesconfirmation to the user on either the instrument cluster or centerstack display of the towing vehicle. The ATA system stores thecalibration values of the trailer into non-volatile memory of thehardware on which the ATA algorithm and software is running. The MVsinclude information about relative data change information, and themaximum correlation matrix for the MVs (MAX_CM_MV) includes informationpertaining to the maximum allowed relative data change at the maximumallowed relative orientation between two sensors of same type.

If the connected trailer has already been calibrated, or after thetrailer has been calibrated, the system will present one display thebird's eye view of “towing vehicle+trailer” in a relative scene map onthe display screen (such as a display screen at a center stack) of thevehicle. The relative scene map is the region behind the towing vehiclewhich covers ‘x’ meters along a longitudinal axis from the center of therear sensor and ‘x/2’ meters at each side of lateral axis from thecenter of the rear sensor. The dimension ‘x’ is preferably greater than2× the length of the trailer, and the dimension ‘x’ depends on imageresolution, field of view and height of the of the trailer sensors. Therelative scene map is generated using the trailer sensors and the multisensor system on the towing vehicle. The ATA system prompts a message tothe user (such as at the center stack display) to select a location inthe relative scene map to which the trailer has to maneuver. The usermay select a target location or point (X, Y) on the center stack displayeither by touching the screen (if the screen is touch enabled) or byusing cursor buttons to move a cursor to the target location or point onthe scene map or by otherwise providing an input indicative of thetarget location.

When the user selects a location on the scene map to which the towingvehicle is to be maneuvered, the selected location in the screencoordinate system is then transformed into real world location (XYZ)using calibration parameters of the sensors. The ATA system detectsavailable free space available in the scene map in which the towingvehicle and trailer can be maneuvered, where the free space is free ofobstacles. The ATA system identifies the path to maneuver towards thedesired location. In a similar manner as shown in FIG. 4, the vehicleand trailer in the scene map is shown in the displayed bird's eye viewimages for viewing by the user. The ATA system initiates the maneuvertowards the desired location once the user agrees with the path plannedby the ATA system. The ATA system stops maneuvering the vehicle andprovides a message to the user whenever an obstacle enters into theplanned path.

When the user selects a location on the scene map, the ATA systemautomatically calculates the distance between the rear end of thetrailer and the user's desired location (D_TrP), the path towards theuser's desired location, and the desired correlation matrix (DCM[ ])between the trailer sensors and the vehicle multi-sensor system forevery 1/500^(th) distance of D_TrP using motion vectors (MVs) of thetrailer sensors and the vehicle sensors. When the maneuver is initiated,the ATA system continuously calculates the current correlation matrix(CCM) at a frequency of, for example, 60 samples per second. If the CCMat a given point of time x is not matching with DCM[x] during themaneuver, then the trailer position will be adjusted by controlling thesteering of the towing vehicle until the CCM matches with DCM values andthe DCM will be updated as per the new position of the trailer. Thisstep continues to happen until the vehicle reaches the desired location.The ATA system considers the direction of maneuver to align the relativedata change rate of the trailer sensors with that of the multi-sensorsystem of the towing vehicle. The system shall not cross the maximumallowed speed at the maximum allowed relative orientation of two sensors(MAX_CM_MV) at any given point of time.

The system may use (as inputs) an on/off vehicle navigation system,which directs the trailer to the destination (see FIG. 9). The systemmay support both on vehicle navigation systems (vehicle-installedsystems) and off vehicle navigation systems, such as a portable or handheld navigation device.

Optionally, the system may use a drone camera for an input. For example,the system may use camera images from external sensor data including adrone camera to create the relative scene map for the trailer maneuver.The drone may park or dock at the vehicle or trailer and may detach forremote image data capture above the vehicle and/or trailer and/orsurrounding area, whereby the captured image data may be wirelesslytransmitted to the vehicle system to assist the system in maneuveringthe vehicle and trailer. Optionally, the drone may, when undocked,remain tethered to the vehicle or trailer and may communicate capturedimage data via a wired connection. The system may utilize aspects of thesystems described in U.S. Publication No. US-2018-0141658, which ishereby incorporated herein by reference in its entirety.

Optionally, the system may use a target based Input, such as an activetarget, where a remote device or transmitter may function as an activetarget. When the user activates the system, it shall track and followthe location of the remote device (see FIG. 10).

Optionally, the system may use a passive target, where the system maytrack any user defined external target. The system may use the target asa reference for the destination location of the trailer. For example,the system may use a user defined painting on a garage wall as areference passive target and, upon detecting it, the system maneuversthe vehicle and trailer towards the target.

The system may allow for a user path selection, where a user can definea path for the trailer to maneuver it towards its destination. Thesystem may map the user defined path to the real world coordinatesystem, and will identify the way points to complete the maneuver.Optionally, the system may allow for an orientation selection, where theuser can identify the orientation of the trailer along with defining theposition and the path. Optionally, the system may be responsive to avehicle-to-vehicle/device communication, where the system may usevehicle-to-vehicle (V2V) or vehicle or other device communication as oneof the input methods to identify the obstacles and free space.

The system of the present invention is suitable for use on all types oftrailers, including trailers attached with a ball/hitch, fifth wheeltrailers, pintle trailers and/or goose neck trailers. The system is alsosuitable for trailers used in the agricultural Industry, used oncommercial trucks, tugs, off-road vehicles (ORVs), in the constructionindustry and in military applications. The system is also suitable fornon-trailer applications, and manufacturing vehicles and applicationswhere multiple trailers are attached to a single towing vehicle.

The system thus automatically guides the trailer that is connected tothe towing vehicle to a user's selected target location. The systemtakes the input from the user to select the destination location andconvert the input data or coordinates to real world coordinates (XYZ).The system then plans the path of the vehicle (and trailer) towards thedestination and maneuvers the vehicle (and trailer) along thedetermined/planned path.

Thus, the trailer maneuvering assist (TMA) system and/or advancedtrailer assist (ATA) system of the present invention operates responsiveto a user selecting or setting a location on the display that representsthe location to which the trailer is to be maneuvered. The systemautomatically calculates an obstacle free path from the initial positionof the vehicle and trailer to the selected location. The systemestimates control parameters at particular intervals required toregulate the trailer for a safer maneuver. The system continuouslyupdates the control parameters during the course of maneuvering andadjusts the path accordingly. The system does not require any sticker ortarget on the trailer. For a trailer setup process, the user drives thetowing vehicle in a forward direction with/without a turn. Whenever anew trailer is connected to the towing vehicle, the user needs to go fora calibration drive to set up the trailer for the vehicle. The systemcontrols the accelerator and brakes of the vehicle while it steers thevehicle to keep the vehicle and trailer moving backward along thedetermined path and toward the selected location. The user still shouldmonitor the display/mirror and apply the vehicle brakes or control thevehicle steering if required, such as to avoid an obstacle or change thepath of rearward travel of the vehicle and trailer.

Optionally, after the user has selected a target or destination locationfor the vehicle and trailer, the user may select a new destination. TheATA system thus may provide an option to the user to change or adjustthe initial/current destination location and/or the orientation of atrailer that is attached to the host vehicle. The ATA system provides anoption to the user to change or adjust the initial destination locationand/or orientation of the trailer with or without stopping the maneuverof the host vehicle. The ATA system may utilize one or more of userinput types, such as a wide variety of input types, foradjusting/changing the destination location.

For example, and such as shown in FIG. 11, the system may provide a dragor touch at the display screen to provide selection of a new destinationor target location. The driver may select a first target location ordestination, and the system may provide for a dynamic (adjusted)destination location. The user is then allowed to change or adjust thedestination location, including the orientation of the trailer that isattached to the host vehicle, at any point of time after activating theATA feature before the trailer reaches the current destination locationor upon the trailer arriving at the current destination location.

Once the user adjusts the destination location and orientation of thetrailer, the ATA system automatically updates the path planner andcalculates the vehicle control triggers required in the new plannedpath. The system uses the current position of the trailer to update thepath planner. Options for the vehicle control triggers to direct thetrailer to the new destination location include (a) in case of fullautonomous driving of the vehicle and trailer, braking control,acceleration control, gear change and steering wheel control, or (b) incase of semi-autonomous driving, steering wheel control only.

The dynamic destination option provides several advantages to the enduser, such as (a) collision avoidance of the trailer with obstacles inthe current path by changing or adjusting the path and (b) adjustment ofthe path when the current path is difficult to maneuver. For example,the following are some of the HMI or control options that can be used bythe user to select or choose the fixed or dynamic destinations. The usermay drag or move or touch the screen for the new destination locationand orientation of trailer on touch screen to change the destinationlocation (see FIG. 11). Optionally, soft or hard buttons may be used tofix/adjust/change the current destination location and orientation ofthe trailer. Optionally, a knob (FIG. 12) may be used to select thefixed or adjusted destination location and orientation of the trailer.Optionally, the steering wheel may be used to select the fixed oradjusted destination location and orientation of the trailer when theATA feature is active.

Referring now to FIGS. 13 and 14, the TMA/ATA feature may use camerasand/or sensors on the towing vehicle and/or trailer to create a surroundand/or three-dimensional (3D) views of trailer backup maneuvers. Forexample, the user may be able control or select a location/destinationand orientation of a virtual camera to provide a useful view on adisplay for the backup maneuver. The user may control the virtual camerathrough a variety of control inputs (e.g., touch screens, knobs, etc.).Optionally, a rearward directed spot light may be used to enhancesensing of the arear rearward of the vehicle and trailer during trailerback-up maneuvers, particularly in low ambient lighting conditions ornighttime driving conditions. The ATA may also use guidelines and/oroverlays (2D or 3D) to further assist the user before and during themaneuver. The ATA feature may provide the views in such a manner thatthe user perceives no blind spots around the towing vehicle and/or thetrailer. The system uses the sensors on the towing vehicle and/or on thetrailer to create the views (2D and/or 3D) during the trailer back-upmaneuvers.

Thus, the user may, while the vehicle is reversing toward a previouslyselected target location or destination, select a new destination and/orselect a new target orientation of the trailer at the destination(initial or newly selected). The system, responsive to the new targetlocation or destination or orientation, adjusts the path of travel ofthe vehicle and trailer accordingly so that the vehicle is controlled tomaneuver the trailer toward the newly selected destination. The systemmay further adjust the path of travel for further selections of targetsor destinations and/or for further selections of targeted or desiredorientation(s) of the trailer at the target or destination.

The system may utilize aspects of the trailering or trailer angledetection systems or trailer hitch assist systems described in U.S. Pat.Nos. 9,085,261 and/or 6,690,268, and/or U.S. Publication Nos.US-2018-0215382; US-2017-0254873; US-2017-0217372; US-2017-0050672;US-2015-0217693; US-2014-0160276; US-2014-0085472 and/orUS-2015-0002670, and/or U.S. patent applications, Ser. No. 16/033,416,filed Jul. 12, 2018 (Attorney Docket MAG04 P3365), Ser. No. 15/928,128,filed Mar. 22, 2018 (Attorney Docket MAG04 P3279), Ser. No. 15/928,126,filed Mar. 22, 2018 (Attorney Docket MAG04 P3277), Ser. No. 15/910,100,filed Mar. 2, 2018 (Attorney Docket MAG04 P3272), which are herebyincorporated herein by reference in their entireties.

The system may utilize aspects of the parking assist systems describedin U.S. Pat. No. 8,874,317 and/or U.S. Publication Nos. US-2017-0329346;US-2017-0317748; US-2017-0253237; US-2017-0050672; US-2017-0017847;US-2017-0015312 and/or US-2015-0344028, which are hereby incorporatedherein by reference in their entireties.

The cameras or sensors may comprise any suitable cameras or sensors.Optionally, one or more of the cameras may comprise a “smart camera”that includes the imaging sensor array and associated circuitry andimage processing circuitry and electrical connectors and the like aspart of a camera module, such as by utilizing aspects of the visionsystems described in International Publication Nos. WO 2013/081984and/or WO 2013/081985, which are hereby incorporated herein by referencein their entireties.

The system includes an image processor operable to process image datacaptured by the camera or cameras, such as for detecting objects orother vehicles or pedestrians or the like in the field of view of one ormore of the cameras. For example, the image processor may comprise animage processing chip selected from the EyeQ family of image processingchips available from Mobileye Vision Technologies Ltd. of Jerusalem,Israel, and may include object detection software (such as the typesdescribed in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, whichare hereby incorporated herein by reference in their entireties), andmay analyze image data to detect vehicles and/or other objects.Responsive to such image processing, and when an object or other vehicleis detected, the system may generate an alert to the driver of thevehicle and/or may generate an overlay at the displayed image tohighlight or enhance display of the detected object or vehicle, in orderto enhance the driver's awareness of the detected object or vehicle orhazardous condition during a driving maneuver of the equipped vehicle.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ladar sensors or ultrasonicsensors or the like. The imaging sensor or camera may capture image datafor image processing and may comprise any suitable camera or sensingdevice, such as, for example, a two dimensional array of a plurality ofphotosensor elements arranged in at least 640 columns and 480 rows (atleast a 640×480 imaging array, such as a megapixel imaging array or thelike), with a respective lens focusing images onto respective portionsof the array. The photosensor array may comprise a plurality ofphotosensor elements arranged in a photosensor array having rows andcolumns. Preferably, the imaging array has at least 300,000 photosensorelements or pixels, more preferably at least 500,000 photosensorelements or pixels and more preferably at least 1 million photosensorelements or pixels. The imaging array may capture color image data, suchas via spectral filtering at the array, such as via an RGB (red, greenand blue) filter or via a red/red complement filter or such as via anRCC (red, clear, clear) filter or the like. The logic and controlcircuit of the imaging sensor may function in any known manner, and theimage processing and algorithmic processing may comprise any suitablemeans for processing the images and/or image data.

For example, the vision system and/or processing and/or camera and/orcircuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641;9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401;9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169;8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935;6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229;7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287;5,929,786 and/or 5,786,772, and/or U.S. Publication Nos.US-2014-0340510; US-2014-0313339; US-2014-0347486; US-2014-0320658;US-2014-0336876; US-2014-0307095; US-2014-0327774; US-2014-0327772;US-2014-0320636; US-2014-0293057; US-2014-0309884; US-2014-0226012;US-2014-0293042; US-2014-0218535; US-2014-0218535; US-2014-0247354;US-2014-0247355; US-2014-0247352; US-2014-0232869; US-2014-0211009;US-2014-0160276; US-2014-0168437; US-2014-0168415; US-2014-0160291;US-2014-0152825; US-2014-0139676; US-2014-0138140; US-2014-0104426;US-2014-0098229; US-2014-0085472; US-2014-0067206; US-2014-0049646;US-2014-0052340; US-2014-0025240; US-2014-0028852; US-2014-005907;US-2013-0314503; US-2013-0298866; US-2013-0222593; US-2013-0300869;US-2013-0278769; US-2013-0258077; US-2013-0258077; US-2013-0242099;US-2013-0215271; US-2013-0141578 and/or US-2013-0002873, which are allhereby incorporated herein by reference in their entireties. The systemmay communicate with other communication systems via any suitable means,such as by utilizing aspects of the systems described in InternationalPublication Nos. WO/2010/144900; WO 2013/043661 and/or WO 2013/081985,and/or U.S. Pat. No. 9,126,525, which are hereby incorporated herein byreference in their entireties.

The system may utilize sensors, such as radar or lidar sensors or thelike. The sensing system may utilize aspects of the systems described inU.S. Pat. Nos. 9,753,121; 9,689,967; 9,599,702; 9,575,160; 9,146,898;9,036,026; 8,027,029; 8,013,780; 6,825,455; 7,053,357; 7,408,627;7,405,812; 7,379,163; 7,379,100; 7,375,803; 7,352,454; 7,340,077;7,321,111; 7,310,431; 7,283,213; 7,212,663; 7,203,356; 7,176,438;7,157,685; 6,919,549; 6,906,793; 6,876,775; 6,710,770; 6,690,354;6,678,039; 6,674,895 and/or 6,587,186, and/or International PublicationNos. WO 2018/007995 and/or WO 2011/090484, and/or U.S. Publication Nos.US-2018-0045812; US-2018-0015875; US-2017-0356994; US-2017-0315231;US-2017-0276788; US-2017-0254873; US-2017-0222311 and/orUS-2010-0245066, and/or U.S. patent application Ser. No. 15/897,268,filed Feb. 15, 2018 (Attorney Docket MAG04 P3267R), which are herebyincorporated herein by reference in their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device, such as by utilizing aspects of the video displaysystems described in U.S. Pat. Nos. 5,530,240; 6,329,925; 7,855,755;7,626,749; 7,581,859; 7,446,650; 7,338,177; 7,274,501; 7,255,451;7,195,381; 7,184,190; 5,668,663; 5,724,187; 6,690,268; 7,370,983;7,329,013; 7,308,341; 7,289,037; 7,249,860; 7,004,593; 4,546,551;5,699,044; 4,953,305; 5,576,687; 5,632,092; 5,708,410; 5,737,226;5,802,727; 5,878,370; 6,087,953; 6,173,501; 6,222,460; 6,513,252 and/or6,642,851, and/or U.S. Publication Nos. US-2014-0022390;US-2012-0162427; US-2006-0050018 and/or US-2006-0061008, which are allhereby incorporated herein by reference in their entireties. Optionally,the vision system (utilizing a forward viewing camera and a rearwardviewing camera and other cameras disposed at the vehicle with exteriorfields of view) may be part of or may provide a display of a top-downview or bird's-eye view system of the vehicle or a surround view at thevehicle, such as by utilizing aspects of the vision systems described inInternational Publication Nos. WO 2010/099416; WO 2011/028686; WO2012/075250; WO 2013/019795; WO 2012/075250; WO 2012/145822; WO2013/081985; WO 2013/086249 and/or WO 2013/109869, and/or U.S.Publication No. US-2012-0162427, which are hereby incorporated herein byreference in their entireties.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

1. A trailer assist system for a vehicle, said trailer assist systemcomprising: a plurality of vehicle sensors disposed at a vehicle, saidplurality of vehicle sensors comprising a rear vehicle sensor disposedat a rear portion of the vehicle and having a field of sensing at leastrearward of the vehicle, a driver side vehicle sensor disposed at adriver side of the vehicle and having a field of sensing at leastsideward of the vehicle, and a passenger side vehicle sensor disposed ata passenger side of the vehicle and having a field of sensing at leastsideward of the vehicle, wherein at least one sensor of said pluralityof vehicle sensors comprises a camera; a trailer sensor disposed at arear portion of a trailer and having a field of sensing at leastrearward of the trailer; a control comprising a processor operable toprocess data captured by said sensors; a display screen disposed in thevehicle and viewable by an occupant of the vehicle, wherein said displayscreen displays images derived from data captured by at least some ofsaid sensors, wherein the displayed images comprise images of at leastan area rearward of the trailer; wherein, responsive to a user inputselecting a target area present in the displayed image, said control,via processing of data captured by said vehicle sensors and said trailersensor, is operable to determine a path of travel for the vehicle andtrailer to follow to maneuver the vehicle and trailer toward a physicaltarget location represented by the selected displayed target area; andwherein said control, responsive to determination of the path of travelwithout further user input, controls at least steering of the vehicle tomaneuver the vehicle and trailer along the determined path of traveltoward the target location represented by the selected displayed targetarea.
 2. The trailer assist system of claim 1, wherein said control isoperable, via processing by said processor of data captured by saidvehicle sensors and said trailer sensor, to detect at least one objectpresent in the determined path of travel.
 3. The trailer assist systemof claim 1, wherein said display screen comprises a touch screen, andwherein the user input is provided by the occupant of the vehicletouching the target area displayed at the touch screen.
 4. The trailerassist system of claim 1, wherein the user input is provided by theoccupant of the vehicle moving a cursor at the display screen to thetarget area displayed at the display screen.
 5. The trailer assistsystem of claim 1, wherein said control estimates control parameters atparticular intervals along the determined path of travel.
 6. The trailerassist system of claim 5, wherein said control updates the estimatedcontrol parameters while maneuvering the vehicle and trailer along thedetermined path of travel and adjusts the path accordingly.
 7. Thetrailer assist system of claim 1, wherein each vehicle sensor of saidplurality of vehicle sensors disposed at the vehicle comprise cameras.8. The trailer assist system of claim 7, wherein said trailer sensorcomprises a camera.
 9. The trailer assist system of claim 1, whereinsaid trailer sensor comprises a plurality of trailer sensors.
 10. Thetrailer assist system of claim 9, wherein at least one of said trailersensors comprises a camera and at least one other of said trailersensors comprises a non-imaging sensor.
 11. The trailer assist system ofclaim 9, comprising a plurality of sensors disposed at a front portionof the vehicle and having a field of sensing forward of the vehicle. 12.The trailer assist system of claim 1, further comprising a remote sensorin wireless communication with said control, wherein said remote sensoris remote from the vehicle and the trailer, and wherein said control isoperable to process data captured by said remote sensor whilemaneuvering the vehicle and trailer along the determined path of travel.13. The trailer assist system of claim 12, wherein the external sensorcomprises a drone sensor disposed at a drone that is operable tomaneuver over the area rearward of the trailer.
 14. The trailer assistsystem of claim 1, wherein, responsive to a second user input selectinga second target area present in the displayed image, said control, viaprocessing of data captured by said vehicle sensors and said trailersensor, is operable to determine a second path of travel for the vehicleand trailer to follow to maneuver the vehicle and trailer toward asecond physical location represented by the second selected target area,and wherein said control, responsive to determination of the second pathof travel, controls the steering and braking of the vehicle to maneuverthe vehicle and trailer along the determined second path of traveltoward the second physical location represented by the displayed secondtarget area.
 15. A trailer assist system for a vehicle, said trailerassist system comprising: a plurality of vehicle cameras disposed at avehicle, said plurality of vehicle cameras comprising a rear vehiclecamera disposed at a rear portion of the vehicle and having a field ofview at least rearward of the vehicle, a driver side vehicle cameradisposed at a driver side of the vehicle and having a field of view atleast sideward of the vehicle, and a passenger side vehicle cameradisposed at a passenger side of the vehicle and having a field of viewat least sideward of the vehicle; a trailer camera disposed at a rearportion of a trailer and having a field of view at least rearward of thetrailer; a control comprising a processor operable to process image datacaptured by said cameras; wherein said control receives a target basedinput from a transmitter remote from the vehicle when the transmitter islocated at a target location rearward of the trailer; wherein,responsive to receiving the target based input that is representative ofthe target location rearward of the trailer, said control, viaprocessing of image data captured by said vehicle camera and saidtrailer camera, is operable to determine a path of travel for thevehicle and trailer to follow to maneuver the vehicle and trailer towardthe target location; and wherein said control, responsive todetermination of the path of travel for the vehicle and trailer tofollow, and without further user input, controls at least steering ofthe vehicle to maneuver the vehicle and trailer along the determinedpath of travel toward the target location.
 16. The trailer assist systemof claim 15, wherein the target based input comprises a communicationfrom an active transmitter that is operable to transmit the target basedinput responsive to actuation of a user input of the active transmitter.17. The trailer assist system of claim 15, wherein the target basedinput comprises a communication from a passive transmitter.
 18. Atrailer assist system for a vehicle, said trailer assist systemcomprising: a plurality of vehicle cameras disposed at a vehicle, saidplurality of vehicle cameras comprising a rear vehicle camera disposedat a rear portion of the vehicle and having a field of view at leastrearward of the vehicle, a driver side vehicle camera disposed at adriver side of the vehicle and having a field of view at least sidewardof the vehicle, and a passenger side vehicle camera disposed at apassenger side of the vehicle and having a field of view at leastsideward of the vehicle; a trailer camera disposed at a rear portion ofa trailer hitched to the vehicle and having a field of view at leastrearward of the trailer; a control comprising a processor operable toprocess image data captured by said cameras; a display screen disposedin the vehicle and viewable by an occupant of the vehicle, wherein saiddisplay screen displays images derived from image data captured by atleast some of said cameras, wherein the displayed images comprise imagesof at least an area rearward of the trailer; wherein, responsive to auser input selecting a target area present in the displayed image, saidcontrol is operable to determine if the system is calibrated for theparticular trailer hitched to the vehicle; wherein, responsive todetermining that the system is not calibrated for the particular trailerhitched to the vehicle, said control calibrates the system for thatparticular trailer; wherein, after the system is calibrated for thetrailer, said control, via processing of image data captured by saidcameras, and responsive to selection of the displayed target area,determines a path of travel for the vehicle and trailer to follow tomaneuver the vehicle and trailer toward a target location represented bythe selected displayed target area; and wherein said control, responsiveto determination of the path of travel, and without further user input,controls at least steering of the vehicle to maneuver the vehicle andtrailer along the determined path of travel toward the target locationrepresented by the selected displayed target area.
 19. The trailerassist system of claim 18, wherein said control is operable to generatedriving instructions for a driver of the vehicle to follow to calibratethe system for the particular trailer hitched to the vehicle.
 20. Thetrailer assist system of claim 18, wherein said control is operable tomeasure motion vectors of at least one of the plurality of vehiclecameras and the trailer camera to calibrate the system for theparticular trailer hitched to the vehicle.